Method and apparatus for data channel augmented auto attended voice response systems

ABSTRACT

Method of using a computerized smartphone to navigate remote auto attendant telephony systems with a menu structure. The auto attendant&#39;s menu structure is put into an online computer database. The caller uses the smartphone to call and establish a voice channel with remote auto attendant telephony system (using the telephone number of that system), software applications running on the caller&#39;s smartphone communication device intercept the telephone number and along with the voice channel, also establish a data channel with the online computer accessible database. The caller&#39;s smartphone retrieves at least some of the menu structure of the auto attendant telephony system through this data channel, and displays at least some of the menu structure of the remote auto attendant telephony system on the graphical user interface of the user&#39;s smartphone synchronized, with the audio delivery of the menu structure, thus facilitating interactions with the auto attendant system.

CROSS REFERENCE TO RELATED APPLICATIONS

This application is a continuation in part of application Ser. No.13/912,422, “METHOD AND APPARATUS FOR DATA CHANNEL AUGMENTED AUTOATTENDED VOICE RESPONSE SYSTEMS”, inventor Millind Mittal, filed Jun. 7,2013, now U.S. Pat. No. 9,112,970; application Ser. No. 13/912,422 was acontinuation of U.S. patent application Ser. No. 12/951,018, “METHOD ANDAPPARATUS FOR DATA CHANNEL AUGMENTED AUTO ATTENDED VOICE RESPONSESYSTEMS”, inventor Millind Mittal, filed Nov. 20, 2010, now U.S. Pat.No. 8,565,388; application Ser. No. 12/951,018 in turn claimed thepriority benefit of provisional application No. 61/361,885 entitled“Method and Apparatus for Augmented Dialing System”, filed Jul. 6, 2010,inventor Millind Mittal; and also priority benefit of provisionalapplication No. 61/361,889, entitled “Method and Apparatus for DataChannel Augmented Auto Attendant and Interactive Voice ResponseSystems”, filed Jul. 6, 2010, inventor Millind Mittal; the contents ofall of these applications are incorporated herein by reference.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The field of the invention is related to automated telephone attendantsystems and telecommunications technology.

2. Description of the Related Art

Over the years automated telephone answering machines, such as automatedattendants, (also called auto attendant (AA) systems, virtualreceptionist systems, interactive voice response (IVR) systems, AA/IVRsystems, and other names) have evolved from relatively simple devices torater complex computerized devices. Modern automated telephoneattendants can, for example, automatically answer telephone calls from auser, play a standard message, and then await a response from theincoming caller. This response will often come in the form of usertelephone key presses, which often generate a standardized set of tonescalled Dual-tone multi-frequency tones (DTMF), or which may come in theform of more sophisticated voice recognition systems that recognizesincoming caller voice input, and can connect the incoming callerappropriately. These auto attendant systems can often have a complexmenu structure, in which an incoming caller from the outside may, forexample, go down a complex tree like sequence of AA/IVR recorded voiceprompts and incoming caller replies in order to connect the incomingcaller with the particular person, departmental function, or informationthat the incoming caller desires.

In this specification, these systems will often be referred to as AA/IVRsystems, or generically often as remote auto attendant telephonysystems, and the incoming caller will occasionally be referred to in thealternative as the caller or the user. The person or information thatthe incoming caller is trying to reach will often be referred to as thereceiver or the callee.

Some of the AA/IVR systems in use today include systems that operateusing voice communication over traditional Public Switched TelephoneNetworks (PSTN), as well as systems that operate over packet networksusing voice-over-IP technologies. Many commercial owners or users ofAA/IVR systems also operate websites which can provide additionalinformation, and indeed often in this case, the organization's AA/IVRvoice prompt will encourage incoming users to stop using the telephoneconnection, and instead switch to using web browser to access theorganization's website for further information.

There are various ways in which an incoming caller can call and interactwith an individual or organization that operates an AA/IVR system and/oran additional website. These include voice only connections, data onlyconnections, and simultaneous but separate voice and data connections.

In a voice only connection, typically the incoming caller will talk tothe callee or receiver using either a conventional voice connection orVoice over IP connection. In both cases, usually a telephone number isused to identify the callee, and the identity of the caller can alsooften be ascertained by using the caller's telephone number. This modeof communication will frequently be referred to as voice channelcommunications. The telephone number used to contact the AA/IVR systemwill often be referred to generically as directory information.

In a data only connection, the caller (or user) will typically not usethe AA/IVR system at all, but rather will interact with the callee (ororganization) through a separate web browser web-site networkconnection. Here the identity of the callee and the caller areestablished through their respective IP address. This mode ofcommunication is one example of data channel communications, but otherdata channel communications that do not rely on web server web browsertechnology are of course also common. Some of these data channel basedinformational sites allow initiating voice communication by clicking onan icon or web link. However, this voice communication is then carriedout as an independent communication.

In a simultaneous but separate voice and data connection, the caller andcallee may interact using both a voice connection for interacting withthe AA/IVR system (and through the AA/IVR system to the callee or thecallee's automated messaging system agent), and also using a datachannel for a web-browser, web-site network connection. In this case,however, both connections will be separate. This experience is probablyfamiliar with any individual who has attempted to place a call to anorganization while simultaneously browsing the organization's website.Here a phone number is used to access the AA/IVR system, while an IPaddress is used to access the website. This mode of communication is oneexample of simultaneous voice and data channel communications. In thiscase, the voice and data channels are driven by separate user inputs.

There are also newer integrated computerized AA/IVR systems thatintegrate both a voice and a data channel to create an integrated voiceand data system, where the caller can provide inputs that only drivevoice, or the caller can provide a combination of inputs for a voice anddata channel. These newer integrated and computerized AA/IVR systems arereferred to as visual IVR systems or visual auto attendant voiceresponse systems. Currently visual IVR systems have had limited successbecause their deployment is both expensive and logistically demanding.These systems require the installation of a completely new visual IVR orvisual auto attendant voice response system on the callee side, as wellas visual IVR capable client software on the caller's smart phone. Inthese newer systems, the systems that handle and transmit visuallyaccessible menu information are logically connected (i.e. often arecontrolled by the same computer processor(s) and execute at least somesoftware code in common) with the systems that handle the remainder ofthe AA/IVR system.

BRIEF SUMMARY OF THE INVENTION

One problem with prior art AA/IVR phone systems is that they generallyonly interact with the caller in the form of voice messages. The AA/IVRphone system will frequently attempt to give a verbal description of arelatively complex sub-menu—i.e. “press 1 if you want sales, press 2 ifyou want customer support, press 3 if you want personnel, press 4 if youwant finance, press 5 if you want our address, press 6 if you want adial by name directory, press 7 to speak to an operator”, rely upon thecaller to remember all this, and then interpret the DTMF tones or voiceresponses sent back by the caller. Present AA/IVR systems do not haveany provision for providing either text or visual information back tothe caller.

As a result, this audio based response limitation leads to a very slowand often frustrating interactive experience. Audio communication bynature is a serial process—only one word can be said at a time, and therate of consumption is limited since keeping the voice legible impliesthat neither the AA/IVR voice verbal description nor the caller responsecan be transmitted at a high rate. By contrast, text or visual promptsand responses are much faster, which is one of the reason why web basedmethods are so popular.

Although some of the newer visual IVR systems can, to some extent,integrate voice and text or visual prompts, as previously discussedthese newer systems are complex and difficult to implement because theyare not backward compatible with audio AA/IVR systems. Rather theyrequire implementation of an entirely new system, which is financiallyand logistically demanding. Thus some sort of method that would improveon prior art and presently deployed AA/IVR systems, without requiring acomplete AA/IVR system upgrade, would be desirable.

The invention is based, in part, upon the insight that if methods couldbe devised to bring some of the enhanced speed and convenience of textand graphic based prompts and responses to prior art audio based AA/IVRsystems without requiring updates to already deployed AA/IVR systems,these systems would become both much easier and faster to use, and alsopotentially represent a new forum in which to bring additionalinformation, commercial messages, and promotional messages to AA/IVRusers. The net result could be a win-win scenario in users of suchenhanced AA/IVR systems would appreciate the increased efficiency andease of use of the system, while the owners or managers of the enhancedAA/IVR system or service could have an additional venue in which tobring additional product, service, or promotional information to theircallers. Such methods of bringing enhanced speed and convenience of textand graphic based prompts and responses to AA/IVR systems, which do notrequire updates to already deployed AA/IVR systems, can also be enhancedover time to provide server side mechanisms for synchronizing voice anddata channels.

This invention is made possible, in part, due to the proliferation of“smart” (computer equipped) telephones, often equipped with an abilityto communicate over both a real or logical voice channel and separatereal or logical data channel. For example smart cellular telephones,such as the Apple iPhone, Google Droid, and other systems now have bothaudio input/output devices (microphones, speakers) as well as highresolution bit-mapped color graphical user displays and interfaces,often equipped touch sensitive screens and either real or virtualkeypads or keyboards. In addition to smart cellular phones, variouswired Voice-over-IP phones also have the ability to establish both voiceand data channel interfaces simultaneously. Here, such smart computerequipped telephones which, in addition to typical phone audio input andoutput devices, also have graphical user interfaces will be referred toas smart phones, and generically as computerized audio communicationdevices, whether they provide cellular, wireless or wirelineconnectivity.

The invention's method will frequently be implemented by a combinationof smart phones, smart phone software (applications, apps), inconjunction with a new type of online (i.e. data network connected,often internet accessible) computer accessible database. This new typeof online computer accessible database will often be logically separate(i.e. not be processed by the same instance of a computer program, alsoreferred to as a process) from the (often legacy) AA/IVR system, andoften will be physically separate (i.e. running on separate processorson equipment that is often physically separated by distances of feet,miles, or even thousands of miles) from the AA/IVR system. The newmethod and system will be typically accessed using a data link that isat least logically separate from the telephony link used to establish acommunication with the AA/IVR system, and often the data link will bephysically separate (i.e. use different frequencies, wires and/ormodulation schemes) from the telephony link with the AA/IVR system.

In one embodiment, the invention may be a method of using a computerizedaudio communication device with a graphical user interface to navigate aremote auto attendant telephony system with a menu structure. Thismethod may consist of obtaining the menu structure informationpertaining to the menu structure of the auto attendant telephony system(e.g. the tree of various potential options open to the user, andvarious responses that the system will accept from the user). Here thismenu structure information will be put into an online computeraccessible database that is often both logically and physically separatefrom the AA/IVR system. When the caller uses his or her computerizedaudio communication device to call and establish a telephony link (i.e.a voice channel or link, or a Voice over IP [VoIP] channel or link, bothpossibilities of which are often simply referred to in the alternativeas the “voice channel”) with a given remote auto attendant telephonysystem (using the telephone number or “directory information” of thatremote auto attendant telephony system), one or more softwareapplications running on the caller's communication device will interceptthe telephone number or directory information, and either prior to,during, or subsequent to establishing the voice channel also establish adata channel with the online computer accessible database. This datachannel will typically be at least logically separate from the voicechannel (audio telephony link), and occasionally this data channel maybe physically separate (i.e. use different frequencies, wires, ortelecommunications protocols) from the audio telephony link as well. Thecaller's communication device can then retrieve at least some of themenu structure of the auto attendant telephony system over a physicallyor logically distinct data channel. This application software can thendisplay at least some of the menu structure of the remote auto attendanttelephony system on the graphical user interface of the user'scomputerized audio communication device (smart phone). As it is commonlyunderstood, Data channel is also referred to as data link throughout theapplication. Similarly voice channel, telephony channel, telephony link,telephony channel are alternatively referred to mean the same element.In the context of current invention it is also the separate link fromthe default telephony link.

As a result, in addition to hearing the remote auto attendant's menustructure through the voice channel's audio channel, the caller can alsosee the same menu structure on the graphical user interface of his orher computerized audio communication device. Since in this case all ofthe available options at a given point in a menu are presentedsimultaneously on the caller device's graphical user interface, thecaller can make an expedited selection of the option he or she desires.Additionally, either through a real keypad, virtual keypad, or by directfinger presses onto a pressure sensitive display, the user can use thegraphical user interface to quickly select the appropriate menuresponse. The application software can then translate this graphicaluser interface selection back to the remote auto attendant by eithergenerating a suitable DTMF or other audio signal on the audio channel,or alternatively generating a different type of signal, such as one ormore symbols to be transmitted back on the real or logical data channelto the online computer accessible database, and from here back to theremote auto attendant. Working with the menu information downloaded fromthe online computer accessible database, and generally navigatingthrough a series of various levels on the remote auto attendant's menuor menu-tree, the user (caller) can navigate through the remote autoattendant menu structure. As he or she does so, the application softwarecan continually adjust and update the display of the menu information onthe graphical user interface on the caller's smart phone. This allowsthe user to track exactly where he or she is in navigating the menustructure of the remote auto attendant telephony system menu.

As will be discussed, in addition to this basic set of features, thismethod and system allows for other information, such as graphics,additional text information, and even promotional messages to also bedisplayed to a remote caller while the caller is interacting with theremote auto attendant telephony system.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 shows an example of a voice connection to a prior art autoattendant telephony system.

FIG. 2 shows an example of a data connection to a prior art computerserver, such as a website or other service supplied by a computeraccessible database.

FIG. 3A shows an example of a prior art dual channel smart phone makinga audio connection with an auto attendant telephony system whilesimultaneously making a separate data connection with a separatecomputer website.

FIG. 3B shows an example of a prior art dual channel smart phone makinga audio connection with an auto attendant telephony system, andsimultaneously making a separate data connection with a visual autoattendant voice response system.

FIG. 4A shows how by using the methods of the invention, the menustructure of an auto attendant telephony system can be relayed to asmart phone by way of a logically separate online computer accessibledatabase and a data channel. This method does not require any newinterface or any change to previously deployed AA/IVR systems. Eventhough the previously deployed AA/IVR system itself has not beenchanged, the caller is now able to navigate through the auto attendanttelephony system's menu structure by way of the caller's smart phonegraphical user interface.

FIG. 4B shows the system described in FIG. 4A, with an optionalsynchronization data channel between the AA/IVR system, and the computeraccessible database.

FIG. 5 shows how the invention may also be used to push additional text,graphics, or promotional materials to the smart phone's graphical userinterface during a call to an auto attendant telephony system.

FIG. 6 shows an example of the phases of a call to an auto attendanttelephony system using the augmented methods of the invention, showingvarious phases where such augmented information can be useful.

FIG. 7 shows an example of the phases of a call to an individual withoutusing either a simplified auto attendant telephony system or no autoattendant telephony system, showing that the augmented connection canalso be used independently to transfer additional data either during orafter a telephone call.

DETAILED DESCRIPTION OF THE INVENTION

As previously discussed, prior art AA/IVR systems, although increasinglyflexible and complex in terms of the provider's ability to craft acustomized response menu for a call originator, are still primarilylimited to only voice based responses. This makes navigating such menusboth complicated and slow. Further, because it is difficult for users toremember complex spoken menus, the operator of the AA/IVR system mustlimit the complexity and number of options provided on the menus.

At the same time, there are a growing number of phones that now have oneor more computer processors, memory, and large display screens withgraphical user interfaces. These include smart cellular telephones,office Voice over IP (VoIP) phones and other modern phones.

Review of Prior Art:

FIG. 1 shows an example of a voice connection to a prior art autoattendant telephony system. This remote system (100) will often consistof a computer processor, software, sufficient memory storage for storingvarious menu prompts and voice mail, sound analyzers (i.e. Analog toDigital (A/D) converters or other devices on the incoming audio channelsto interpret incoming DTMF signals and/or user voice commands), andgenerally a plurality of various telephone connections (102), (104) thatconnect to the various individuals and departments served by thatparticular auto attendant telephony system. The auto attendant telephonysystem will have a menu structure (106), typically residing in memory,that is composed of at least the various prompts and caller inputs thatthe system recognizes, along with various ancillary information such asdelay times, audio prompt signatures, and the like.

In use, a caller using a phone such as a smart phone (110), which may beequipped with a graphical user interface (112) (here a bit mapped, touchsensitive graphical display screen) and a real or virtual (i.e. createdby a software controlled graphical display of a keypad and a touchsensitive screen) keypad (114) as well as a microphone (116) and speaker(118) can call the auto attendant telephony system using a real orlogical audio channel (120) to provide voice inputs and commands. Whenthe caller calls the auto attendant telephony system (100) over voicechannel (120), the system will usually answer with a voice message (122)that recites at least the top portion of the system's menu (106), andthe caller will hear this message over speaker (118). Once a voicechannel (120) is active, selection of any key on keypad (114) results increation of a DTMF tone (124) to send to auto attendant telephony system(100) over voice channel (120). Auto attendant telephony system (100) onreceiving the DTMF input, then navigates to the next level in menu(106). For example, DTMF signals corresponding to selection of key “2”will result in menu (106) to proceed to the next level corresponding tothe customer service options.

FIG. 2 shows an example of a data connection to a prior art networkconnected computer server, such as an Internet website or other servicesupplied by a computer accessible database. Prior art computers andsmart phones are also often equipped with network data connections, suchas wired or wireless Internet connections (200) that enable the computeror smart phone to establish a physical or logical data channel or linkwith remote network servers (equipped with computer processors, softwareand memory), such as web servers (202), which in turn have computeraccessible databases (204) that may serve up web pages and other usefulinformation. Using a prior art website as an example, a user of a smartphone (110) equipped with a web browser (210) running in the smartphone's graphical display screen (112) may display information from thesame organization that is running the auto attendant telephony system(100) shown in FIG. 1, or a different web server, and interact with thatwebsite by, for example touching various web browser links and menuoptions on a touch sensitive graphical user interface screen (212), orby other methods (e.g. mouse, pointer, keypad, keyboard, etc.), creatingthe associated request (214) and communicating the request to the webserver over the data channel or link (200). The method describing thisprior art may employ various known caching techniques such that a numberof requests may be satisfied within smart phone.

FIG. 3A shows an example of a prior art dual-channel smart cellularphone making a audio connection with an auto attendant telephony system,while simultaneously making a separate data connection with a separatecomputer website. This prior-art example is essentially equivalent toFIG. 1 and FIG. 2 combined. Although the caller may be in simultaneouscommunication with an organizations AA/IVR system (100), and with thesame organization's web site (200), (204), there is no synergy. That is,the organization's website will not help the caller navigate the AA/IVRsystem. The two functions act independently.

FIG. 3B shows an example of a prior art dual-channel smart cellularphone making an audio connection with a visual auto attendant telephonysystem. In this prior art example, simultaneous voice channel (120) anddata channel (200) interaction requires, in addition to a new smartphone application (112), replacing auto attendant voice response system(100) with a new class of visually enhanced auto attendant voiceresponse system (130), using an enhanced navigation menu (136). Inputsto visually enhanced auto attendant are generated by selecting keypad(104) and/or visual menu and other information (e.g. extra text,graphic, or even video) information (408). Depending upon theapplication, selected inputs can be provided to visually enhanced autoattendant over voice and/or data channel.

In the case of at least smart phones exemplified by the Apple iPhone andthe Google Droid series of smart phones, it is very easy to add newsoftware applications or “apps” to add additional functionality to thesmart phones by downloading these applications from various onlineapplication stores or repositories. These applications allow both thefunctionality and the user interface of the smart phones to be enhancedor altered according to that particular application.

FIG. 4A illustrates one embodiment of the invention's data channelaugmented auto attended voice response system. In this case (either byone or more software programs preloaded by the manufacturer of the smartphone, or by a downloadable application) the basic functionality of thesmart phone can be upgraded. Using this upgraded functionality, when theuser (caller) dials the telephone number of another telephone, the smartphone's software can intercept this telephone number (here calleddirectory information), and establish a data channel or link with alogically separate online computerized accessible database. The smartphone can then use the dialed telephone number as an index or search keyinto this database (402), and if the database knows that the phonenumber corresponds to an auto attendant telephony system, and also knowsthe menu for that auto attendant system, then the database can quicklysend this menu to the smart phone by the data channel. This process ofusing the phone number (directory information) as an index to a databasethat may contain menu information or additional information pertainingto the number that is being called is here termed “augmented dialing”.

Using augmented dialing, for example, the graphical user interface forthe smart phone may still show a phone number keypad (114), but sincethe smart phone is also getting menu and other information (e.g. extratext, graphic, or even video) information (408) back from the onlinedatabase through the data channel (200), then the graphical userinterface can also show quite a bit more.

Thus in one embodiment, the invention can be a method of using acomputerized audio communication device with a graphical user interface,such as a smart phone, to navigate a remote auto attendant telephonysystem (i.e. an AA/IVR system) with a menu structure.

This method depends upon first obtaining the menu structure of theremote auto attendant telephony system, and putting this menu structureinto an online computer accessible database. The menu structure may beobtained in various ways. It may be directly provided by the operator ofthe auto attendant system (100). Alternatively, an outside service maymanually query the remote auto attendant telephony system, and producethe menu structure that way. As yet another alternative, an automatedsystem may crawl the remote auto attendant telephony system using voicerecognition algorithms, using techniques not unlike those of automatedweb crawling robots. Whatever way the menu structure is put into thedatabase, the database can be designed so that inputting the telephonenumber (directory information) of that particular automated autoattendant system (100) into the database will act to retrieve theappropriate menu structure (106) information, along with extrainformation (i.e. extra text, graphics, promotional information) as willbe discussed.

When the smart phone (110) establishes a data channel (200) with theonline computer accessible database, it will pass the phone number thatthe user just dialed to the database (402), and assuming that thisinformation is in the database, the database will then retrieve the menustructure information for that particular automated auto attendant (andother associated information as well) and pass it back to the smartphone using data channel (200). The software application running on thesmart phone can then take this menu structure information and displayrelevant portions of it on the smart phone's graphical user interface(408). The smart phone user (caller) can then use this displayed menustructure information to guide navigation through at least some of theremote auto attendant telephony system's menu structure.

Thus, for example, if the menu structure of the AA/IVR system at aparticular point presented the user with options such as: “Press 1 forsales, press 2 for customer service, press 3 for billing, press 4 forthe operator”, then in addition to hearing this portion of the menu fromthe AA/IVR system in the form of a vocal prompt, the user or callerwould also see this on the smart phone's graphical display in writing(408). Depending upon the application implementation, the user or callermay provide inputs in one of several different ways. The caller couldsimply respond to the AA/IVR system by either pressing keys on a real orvirtual keypad (114), directly interact with the menu selection (408)that is displayed on the smart phone's graphical user interface (112),or may elect to speak the menu selections into the smart phone'smicrophone (116). In each of the approaches for providing input tonavigate the menu, for some of the menu traversal steps, the selectedinput is used to both navigate through AA/IVR system, and to navigatethe display menu structure.

For the case when the input is provided by pressing keys on either areal or virtual keypad (114), for some of the menu traversal steps, theselected input may generate both DTMF audio signals (412) back to theAA/IVR system over the voice channel (120), and also a data request(406) back to the server (400) to drive the menu structure information.

For the case when input is provided by interaction with the menuselection (408), for some of the menu traversal steps, the applicationsoftware may provide a mapping of the selected input to one of the keysof keypad (104), and generate a corresponding DTMF audio signal (412)back to the AA/IVR system over the voice channel (120), as well as adata request (406) back to the server (400) and database (402) torequest further menu structure information, and to drive the menustructure information. In another embodiment, instead of mapping theselected input to a corresponding DTMF audio signal, the applicationsoftware may map the input to a predefined non-DTMF audio waveform (412)that corresponds to the same navigation control and is understood by thespeech recognition processing, if any, for AA/IVR system. In thisexample, the user input might be mapped onto a corresponding Englishphrase that can be sent to the voice recognition section of the AA/IVRsystem. This embodiment may be particularly useful for callers withspeaking impediments.

In the situation where the caller provides input to the AA/IVR system byspeaking the selection into the microphone (116), for some of the menutraversal steps, spoken input, in addition to being sent over voicechannel (120), is may be mapped to corresponding data request (406),sent to server (400) and database (402), and also used to drive requestsfor further menu information, as well as other desired functionality.

In all of the above three input cases, if the data request is notlocally satisfied through local memory storage on device (110), therequest may be sent to external server (400) and database (402) overdata channel (200). Note that for both non audio input mechanisms, i.e.keypad based (114) or display menu selection based (408) mechanisms, thestep of creating the corresponding audio signals is shown as step (412)in FIG. 4A and FIG. 4B.

Note that according to the invention this additional ability to navigateby a graphical user interface synchronized with the AA/IVR may be madepossible without making any changes to the AA/IVR system whatsoever.This method thus has the substantial advantage of being backwardcompatible with the hundreds of thousands or millions of previouslydeployed AA/IVR systems in the field.

Once the user has made a particular menu selection, the remote AA/IVRsystem (100) will move on to a different part of its menu, and theapplication software running on the smart phone, knowing the menuselection and the structure of the AA/IVR menu, can update the displayof the menu information on the smart phone's graphical user interface tobetter track the current status of the AA/IVR (remote auto attendanttelephony system).

Although during the navigation through the menu structure of AA/IVR,during this discussion it has been often assumed that the smart phone(computerized audio communication device) is capable of operating both avoice channel and a data channel, and that the menu structureinformation is obtained through the data channel, other modes ofoperation are also possible. In particular, although the data channel(200) may operate simultaneously with (at the same time as) the voicechannel (120), this is not always necessary. For example, right afterthe user enters in the phone number, but before the voice channel isactivated, the smart phone might quickly use the data channel (200) tointerrogate the online computer accessible database and find out if thetelephone number corresponds to a known AA/IVR system with a known menustructure. If not, then the data channel may not be used further, andthe telephone can then put through the call to the other phone numberusing the voice channel. Alternatively, the online computer accessibledatabase may inform the smart phone that the AA/IVR system menu hasvarious hold periods during which additional information may beretrieved from the online computer accessible database over the datachannel. As yet another alternative, the online computer accessibledatabase may inform the smart phone that additional information relevantto the AA/IVR system (such as promotional coupons) is available aftertermination of the voice channel. This way, the smart phone can manageits use of the data channel, and reduce its power and data useexpenditures accordingly.

Not all smart phones are equipped with both long distance voice channelsand long distance data channels. Rather, in some embodiments, some smartphones (e.g. smart cell phones) may be equipped with a single longdistance voice channel and instead have only a short distance datachannel, such as a Bluetooth™ or Wi-Fi data channel available for datause. As long as there is a suitable wireless router nearby capable ofpicking up the short range data channel and relaying this to and from amore distant online computer accessible database, then the invention cancontinue to function adequately.

In a somewhat similar vein, some smart phones may communicate voiceusing voice over IP (VoIP) technology. In a situation like this, thevoice over IP channel may be treated as a logical voice channel becauseusually there is an intermediate apparatus between the voice over IPsmart phone and the remote automated attendant (AA/IVR) system thattranslates the voice over IP signal back into an audio signal. In asituation like this, a smart phone may employ the methods of thisinvention by communicating with the AA/IVR system using one logicalvoice over IP voice channel, and also communicate with the onlinecomputer accessible database using same IP channel as a logical datachannel.

In some embodiments, even though the smart phone is capable ofestablishing a cellular or PSTN voice channel, in some cases, such as toreduce phone company charges for certain types of telephone numbers, theaugmented dialing system may alternatively utilize VoIP technology forthe voice channel. This use of VoIP technology for the voice channel mayalso be useful in other situations as well. For example, some smartphone platforms have built-in manufacturer limitations or constraintsthat are designed to preclude 3^(rd) party application software fromdirectly interfacing with the smart phone's voice channel. Here toovercome this manufacturer constraint, the augmented dialing system mayalternatively utilize VoIP technology for the voice channel.

FIG. 4B adds a synchronization data channel (404) between the server(400) and the auto attendant telephony system (100). The system shown in4B is generally similar to the system shown in FIG. 4A 400, except thatin this case, data channel 404 can be utilized to achieve an addedsynchronization of voice and data channel. This can be particularlyuseful in situations where the AA/IVR menu structure requires that thecaller provide additional non-menu input (such as caller identificationinformation) that does not relate to the navigation of the AA/IVR menustructure. When such non-menu input data is required, the normal menuprogression is interrupted because the auto attendant telephony system(100) will typically need to validate whether the combination of inputsprovided by the caller is valid or not, before menu navigation canprogress further.

In the system described in FIG. 4A, such identification input, alongwith the subsequent need to validate the identification input, may causeinterference with normal menu navigation. In the case where data channel(404) is present, data channel (404) can be utilized to communicate theoutcome of the validation step, and thus automatically synchronize voicechannel (120) and data channel (200) to the next step in the menustructure.

In the case where synchronization data channel 404 is disabled, thesystem defaults back to methods previously described in FIG. 4A. In thiscase handling of non-menu input data may be done by an additionalsynchronization step within the augmented dialing application on thesmart phone. This synchronization step may be achieved by the callerproviding an additional explicit input to indicate that the non-menuinput data step has been successfully completed, and that the visualmenu can also be now navigated to the next step. Alternately, thissynchronization step may be achieved by an augmented dialing applicationon the smart phone performing audio processing of the audio message sentby AA/IVR system indicating successful completion of the non-menu inputstep.

In FIG. 4B, if the synchronization data channel (404) is present, thedirection of data flow may occur primarily between the menu structurememory (106) of an auto attendant telephony system (100), and thecomputer accessible database (402). However in some embodiments, thedata may also be sent back from the computer accessible database (402)to the menu memory (106). In this back migration scheme, a user mayalternatively enter auto attendant menu instructions into their smartphone, and have these instructions be sent to the server (400) andcomputer accessible database (402) by the data channel (200) rather thanby the audio channel (120).

As previously discussed, in addition to transmitting AA/IVR menustructure data, the online computerized database (402) may also be usedto store and transmit additional information. One particularly usefultype of additional information is promotional information. Thispromotional information may contain, for example, graphic images of theproducts or services offered by the sponsor or owner of that particularAA/IVR system, as well as text, sound, or even video promotionalmaterial. Another type of promotional material may consist of couponsthat may be sent to the smart phone upon selecting certain AA/IVR menuselections, or upon completion of the voice channel. Thus for example, acaller to a particular business might be rewarded with coupons at theend of the call. Other information can include material such as locationmaps, and other useful information.

This promotional material (500) may be put onto the online computeraccessible database (402) in various ways. Here again, the owner of theAA/IVR system may simply collect and send the promotional material tothe operator of the database, or alternatively a third party may alsocollect and provide this material. As yet another alternative, thispromotional material may be automatically or semi-automatically acquiredby a web crawling software “robot”, which might for example simply visitwebsites associated with the owner or operator of the AA/IVR system, andproduce promotional material based on data from this website.

FIG. 5 shows how the invention may also be used to push additional text,graphics, or promotional materials to the smart phone's graphical userinterface (112) during a call to an auto attendant telephony system(100). This embodiment is generally similar to that of FIG. 4A, with theexception that in addition to storing the menu structure (106) of autoattendant telephony system (100) in computer accessible database (402),database (402) may additionally store other information, such as in thiscase, graphical promotional information (500). Thus for example, if theauto attendant system (100) is run by an auto dealership, thepromotional materials may include graphic images of the dealership'scars. When smart phone (110) downloads menu information from computeraccessible database (402) over data channel (200), the promotionalmaterials (500) are also transmitted over the data channel, and thesmart phone's application software can display the materials (502) alongwith the display of the menu structure.

These promotional materials can be input into the database (402) by anetwork connection (504) or by a wide variety of alternative methodssimilar to the methods (transfer of memory media, web crawling, thirdparty service) previously used to transmit the menu structureinformation. Here the term “promotional materials” is intended toencompass any additional information beyond that of the basic menustructure, timing, and audio prompt information utilized by the autoattendant system (100). Thus maps, addresses, news items, video, audio,or any other media may also be sent by this method. Again note that thisinformation need not be displayed in a web browser on smart phone(110)'s graphical display (112), but rather may be displayed by theapplication software working, as appropriate, with the various smartphone graphical API.

In order to help synchronize the display of the menu of the remoteAA/IVR system with the display of its status on the graphical userinterface of the caller's smart phone, often it will be useful toinclude information on the AA/IVR menu timing delays as part of the menustructure that is stored in the online computer accessible database.Thus if, for example, if an AA/IVR system is designed with a menu thatbecomes responsive to user input only after the AA/IVR system plays a 10second audio prompt, then this time delay may be included as part of themenu structure information. The user's smart phone can then read thistime delay information from the menu structure data that it obtains fromthe computerized database, and update its display accordingly.

In some embodiments, it will be useful to design the smart phone'sapplication software with an ability to monitor the audio channel foruseful incoming audio cues from the AA/IVR system, and also embedinformation pertaining to these incoming audio cues as part of theAA/IVR menu structure that is uploaded to the smart phone. For example,to improve synchronization between the smart phones display of thestatus and current menu prompt of the AA/IVR system, and the actualstate of the AA/IVR system, the smart phone application software may bedesigned to monitor the audio input, and detect characteristic sounds,sound lengths, and periods of silence that characterize each level ofthe AA/IVR menu. This pattern of sounds, sound lengths, and periods ofsilence can also be included along with the menu structure data anduploaded to the smart phone. Thus if, for example, a particular voiceaudio prompt from a particular part of the AA/IVR menu such as: “ . . .pause 4 “press five to talk to Steve Smith”, pause 5, “press six to talkto Jane Yu”, the application software can analyze the AA/IVR responsefor, example, the “s” sounds characteristic of “Steve Smith”, as well asthe lengths of pause 4 and pause 5, verify that the smart phone is infact properly synchronized with the remote AA/IVR system, and if notmake appropriate corrections. The application software can also monitorfor presence of DTMF tones from the AA/IVR system or remote callee, anduse this information to achieve proper synchronization with the remoteAA/IVR or callee accordingly.

Using these timing and audio interpretation methods, the applicationsoftware running on the smart phone may additionally monitor the audioconnection from the AA/IVR system, and make adjustments to the displayof the AA/IVR menu structure on the smart phone's graphical userinterface in other ways as well. For example, the application softwaremay monitor the audio sound from the AA/IVR systems for sound patternstypically emitted by that particular AA/IVR system when the incomingcaller is put on hold, and display a suitable text or graphic “on hold”message when this occurs. Similarly when the application softwaredetects sounds from the AA/IVR system characteristic of a “hold” ending,this software can again update the smart phone's graphical displayaccordingly. This way the graphical user interface is always properlysynchronized and determined by the current status of the smart phone'svoice channel with the AA/IVR system. The application software will alsomonitor the user smart phone input as well, and use this input asanother way to keep the display of the AA/IVR menu synchronized with theactual status of the remote AA/IVR system. This synchronization processcan also be termed an “adjust” and/or “track” process.

FIG. 6 shows an example of the phases of a call (or alternatively as thestatus of the voice channel), to an auto attendant telephony system suchas (110) using the augmented methods of the invention, showing variousphases where such augmented information can be useful. FIG. 6 shows thetimeline (600) for such a hypothetical call.

The call may start (602) when the caller starts entering a telephonenumber (directory information) onto a real or virtual keypad (114).During this period of time (1) (604), the complete telephone number isnot yet known to the smart phone and to the smart phone applicationsoftware. During this period of time (1) (604), often the smart phonemay show a relatively neutral visual display. Once the complete phonenumber is known to the smart phone augmented or enhanced dialingapplication software (606), the smart phone may open a data channel(augmented connection) (200) to the computer accessible database (402),and start to retrieve menu structure information (106) and otherinformation either quickly, or alternatively throughout the duration ofthe phone call and even beyond as is shown in (608). The smart phonewill also start connecting to the auto attendant system (100) (606).During this time period (2) (606), either as soon as menu data starts tocome in from database (402), or even sooner if the menu data waspreviously retrieved and stored in a memory cache onboard the smartphone, the smart phone can start to display various pre-messages on itsgraphical display (112). For example, if the user is connecting to anauto dealership, the smart phone may display an image of the cars soldby the auto dealership (502).

Once the smart phone connects with the auto attendant system (100),(610), the auto attendant system (100) can start sending an audio menuselection (122) to the smart phone by the voice channel or link (120).This occurs during time period (3) (612). During this time, the systemmay display various menus on the smart phone's graphical display (112).

In this example, assume that the callee is talking on another phoneline, and the auto attendant system gives the caller the option to beplaced on hold for an undetermined period of time (4) (614). Here theapplication software running on the caller's smart phone, in addition tomonitoring user input from the smart phone to the auto attendant system,may optionally also monitor the incoming audio signal from the autoattendant machine (100) for sounds that are characteristic of the holdprocess. During the hold period (614), which otherwise would essentiallyrepresent wasted time, the invention may take advantage of theopportunity to display promotional material (500), (502) or otherinformation. Here for example, an auto dealership may show images andpricing information on various sales that are currently underway.

Once the caller reaches the callee in step (5), the application softwarerunning on the caller's smart phone may again function to detect thechange in audio signal associated with the end of the hold process. Thecaller and callee can then communicate through the normal audio channelduring this time. However according to the invention, another method ofcommunication can also be utilized. Here, for example, if the calleewishes to push additional information to the caller, (618), the calleemay send the desired information to database (402) by network connection(404), and this in turn may be relayed to the caller's smart phone byway of server (400), and data channel (200).

Once the caller and callee terminate their conversation and hang up(620), there is yet a final post-termination phase of the phone call (6)(622) that can also be utilized according to the invention. Here afterthe voice channel (120) but possibly not before the data channel (200)is terminated, the data channel may be utilized to transmit additionalpromotional material, such as coupons and the like, to the caller.

In FIG. 6, the time periods of operation of the auto attendant system isshown as (607) (here the auto attendant may be operating in thebackground during the last part of the call) and the period of time ofoperation of the augmented or enhanced dialing connection and datachannel is shown as (608).

As previously discussed, nearly all cellular telephone users, and evenmany land line users, are currently connected to at least simple autoattendant systems by way of their telephone carrier.

Although the case where the remote auto attendant telephony system is anauto attendant telephony system (AA system) or an interactive voiceresponse system (IVR system) has been frequently used in thisspecification, other types of remote auto attendant telephony systemsare also possible. For example, it has now become standard in the cellphone industry, as well as very common for land line phones as well, toroute an incoming caller to an automated mailbox, which may becontrolled by the cell phone or land line phone carrier. This situationmeets the criteria of an auto attendant telephony system as well,because such standard answering systems have both audio prompts and atleast simplified menus. Thus the methods of the present invention can beused even when the callee is simply a private individual with a cellphone or land line who is using a standard cell phone carrier or landline carrier answering service. In this specification, whenever anAA/IVR system is used as a specific example of a type of remote autoattendant telephony system, the example should be considered to extendto all types of remote auto attendant telephony systems.

FIG. 7 shows how the invention may be highly useful even in thesituation where a caller is using a smart phone to call a callee who mayonly ostensibly have a cell phone and land line, and may not have anobvious auto attendant system.

In FIG. 7, a smart phone caller is calling an individual. Here asbefore, during the first phase of the connection (1) (700), the phonenumber of the callee may not be known, and during this time, often theinvention will show a neutral or general display on the user's smartphone. During the second phase of the connection (2) (702), the callee'sphone number will now be known to the smart phone's applicationsoftware, and according to the invention, a data channel (200) may beopened to the computer accessible database (402), allowing menustructure information (106) to be downloaded to the smart phone (110).Alternatively this information may be already residing in the smartphone's memory cache from a previous call. In the case where the calleeis an individual, the menu structure information may additionallyinclude information pertaining to the capabilities of the callee'stelephone. For example, if the callee's telephone is a smart phone thatitself is equipped with suitable software to receive augmented data ondata channel (200), then this fact may be made known to the caller aspart of the menu structure information.

In the case where the caller is quickly connected to the callee (3)(704), then the conversation may proceed on the audio channel (120) asnormal. However in the case when the callee is unavailable, and theservice provider puts the call into a voice mail system (706), thenaccording to the invention, the caller may have an additional option notnormally available, which is that if the callee has a suitably capablesmart phone system, the caller may elect to push (708) information (suchas a message, graphics, voice, etc.) to the callee by way of datachannel (200), server (400), database (402) and network connection(404).

According to the invention, there are other opportunities for enhancedor augmented data exchange as well. For example, during the call, thecallee may also push (710) information to the caller as well by way ofnetwork connection (404), server (400), database (402) and data channel(200). Even after the voice conversation has ended (4) (712), thecallee, for example, may elect to push additional data to the caller(714).

FIG. 7 shows an example of the phases of a call to an individual withoutusing either a simplified auto attendant telephony system or no autoattendant telephony system, showing that the augmented connection canalso be used independently to transfer additional data either during orafter a telephone call.

Alternate Embodiments

In addition to some of the alternate embodiments already described inthe main text, several other embodiments are also possible. For example,in an alternative embodiment of the invention shown in FIG. 4A, voicechannel (120), instead of connecting to the Auto attendant telephonysystem (100) directly, may alternatively connect through other types ofbridging solution such as hosted VoIP systems. In another alternateembodiment, the smart phone may be a wireline smart phone with embeddedprocessing capability, a graphical user interface, and an ability toestablish one or more data channels.

Note further that the user interface display graphics shown in FIGS. 1,2, 3, 3A, 4, 4A, 4B, FIG. 5 are for illustration purposes only. Manyother different types of related user interfaces and user interfacedesigns are also possible within the scope of the invention.

In another alternate embodiment, the smart phone may exist in the formof telephony application (software) for any internet connectedcomputerized device, such as a desktop computer, notebook computer,netbook, or tablet computer.

In some embodiments the smart phone may implement the invention as anapplication running on an embedded processor within an internetconnected wireline telephone with a visual display and a virtual orphysical keyboard and capability.

In some embodiments the smart phone may implement the invention asfirmware running on an embedded computer processor. In otheralternatives it may be implemented as hardwired state machine in anembedded computer controller.

Any advantages and benefits described may not apply to all embodimentsof the invention. When the word “means” is recited in a claim element,Applicant intends for the claim element to fall under 35 USC Sect. 112,paragraph 6. Often a label of one or more words precedes the word“means”. The word or words preceding the word “means” is a labelintended to ease referencing of claim elements and is not intended toconvey a structural limitation. Such means-plus-function claims areintended to cover not only the structures described herein forperforming the function and their structural equivalents, but alsoequivalent structures. For example, although a nail and a screw havedifferent structures, they are equivalent structures since they bothperform the function of fastening. Claims that do not use the word“means” are not intended to fall under 35 USC Sect. 112, paragraph 6.Signals are typically electronic signals, but may be optical signalssuch as can be carried over a fiber optic line.

The foregoing description of the embodiments of the invention has beenpresented for the purposes of illustration and description. It is notintended to be exhaustive or to limit the invention to the precise formdisclosed. Many modifications and variations are possible in light ofthe above teaching. It is intended that the scope of the invention belimited not by this detailed description, but rather by the claimsappended hereto.

1. A method of using a computerized audio communication device with agraphical user interface to navigate a remote auto attendant telephonysystem with a menu structure, said method comprising; using saidcomputerized audio communications device to establish a telephony linkwith said auto attendant telephony system and a separate link with alogically separate online computer accessible database comprising menustructure information; retrieving at least some of said menu structureinformation from said logically separate online computer accessibledatabase; using said graphical user interface to navigate through atleast some of said remote auto attendant telephony system's said menustructure, wherein said computerized audio communication device adjuststhe display of at least some of said menu structure information on saidgraphical user interface to track the current status of the remote autoattendant telephony system menu.
 2. The method of claim 1, wherein saidcomputerized audio communication device simultaneously operates a voicechannel and a separate data channel, and said menu structure informationis obtained through said data channel.
 3. The method of claim 2, whereinsaid computer accessible database is interrogated using a directoryinformation to determine if said data channel should be activated beforeactivation of, during or after termination of said telephony link. 4.The method of claim 3, wherein said directory information comprises atelephone number, and said telephone number is also used determine howto access said online computer accessible database.
 5. The method ofclaim 2, wherein said computerized audio communications device is asmart cellular telephone.
 6. The method of claim 5, wherein the steps ofdisplaying at least some of said menu structure information on thegraphical user interface of said computerized audio communicationsdevice are performed by a downloadable software application.
 7. Themethod of claim 5, wherein said data channel is a Wi-Fi data channel ora cellular telephone data channel.
 8. The method of claim 5, whereinsaid smart cellular telephone communicates with said remote autoattendant telephony system via either a voice channel or a Voice over IPdata channel.
 9. The method of claim 1, wherein said menu structureinformation additionally comprises promotional information.
 10. Themethod of claim 9, wherein said promotional information is derived bycrawling through at least one Internet website.
 11. The method of claim9, wherein either prior to, subsequent to termination of, or duringinactive periods of said telephony link with said remote auto attendant,said display of at least some of said menu structure information on saidgraphical user interface is determined by the current status of thetelephony link.
 12. The method of claim 1, wherein during some of thetime while said telephony link is active; said computerized audiocommunication device utilizes audio telephony information from saidtelephony link to determine what portion of the auto attendant telephonymenu structure is currently being utilized.
 13. The method of claim 12,wherein Dual-tone multi-frequency (DTMF) tones or voice recognition isused to determine what portion of the auto attendant telephony menustructure is currently being utilized.
 14. The method of claim 1,wherein said remote auto attendant telephony system is an auto attendanttelephony system or an interactive voice response system.
 15. The methodof claim 1, wherein the same user input into said graphical userinterface is used to both navigate through said remote auto attendanttelephony system, and to provide input to said computerized audiocommunication device to adjust the display of at least some of said menustructure information on said graphical user interface to track thecurrent status of said remote auto attendant telephony system menu. 16.The method of claim 1, wherein during at least some of the time whilesaid telephony link is active; said computerized audio communicationdevice utilizes audio telephony information from said telephony link todetermine what portion of the auto attendant telephony menu structure iscurrently being utilized.
 17. The method of claim 1, wherein saidcomputerized audio communications device is an Internet connectedcomputerized device, such as a desktop computer, notebook computer,netbook, or tablet computer.
 18. A method of using a computerized audiocommunication device with a graphical display and voice commandsinterface to navigate a remote auto attendant telephony system with amenu structure, said method comprising; using said computerized audiocommunications device to establish a telephony link, also referred to asvoice channel, with said auto attendant telephony system and a separatedata link, also referred to as data channel, with a logically separateonline computer accessible database comprising menu structureinformation; retrieving at least some of said menu structure informationfrom said logically separate online computer accessible database; usingsaid voice command interface over the said voice channel to navigatethrough at least some of said remote auto attendant telephony system'ssaid menu structure, wherein said computerized audio communicationdevice adjusts the display of at least some of said menu structureinformation on said graphical display interface to track the currentstatus of the remote auto attendant telephony system menu.
 19. Themethod of claim 18, wherein said computerized audio communicationsdevice is a smart cellular telephone.
 20. The method of claim 18,wherein said menu structure information additionally comprisespromotional information.